Occluded Hand Detection for One-view-based Sign Language Interpretation System

Communication between human and hearing impair persons is very difficult in daily life. Sign language interpretation system can translate signs into text or voice for solving this problem. In practically, however, it is not convenient because it cannot be carried by the signer. It is better if the hearing impair persons can bring the sign language interpretation with themselves all the times such as fixed small camera attached to the signer shirt. The acquired image from this camera is back-hand-viewed. One important problem of the back-hand image analysis is an occlusion. This paper presents a new method to recognize the back-hand images with occlusion problems. This approach consists of three steps: finger detection, tracking system and distance measurement. After input image is captured, the system will find the reference finger positions by finger-angle computation. Real-time tracking algorithm is used for checking occlusion status. Finally the system will find the occluded position from similarity measurement and match them with a sign database for display the meaning of the sign. Experiments show that the proposed method yields highly accurate recognition under simple implementation condition. Index Terms – Hand detection, object tracking, hand posture, hand gesture recognition, sign language recognition